====================== 
SUMMARY: 
====================== 

 - The ESP-r 'reference' platform is Linux / GCC 4.1.2
 - We are no longer thoroughly testing GCC3
 - GCC 4.2.3 builds are not as robust
 - GCC 4.3 and 4.4 have not been tested
 - NRCan is no longer conducting cross-platform and cross-compiler tests 
   (See Portability Testing)
  
Recommendations: 
 - Proceed with release 11.8 
 - Discontinue support for gcc 3.4.6 as soon as stable versions of 
   GCC4 are available on all platforms.
 - GCC 4.2+ and Intel builds should be used with caution based on
   experience from the previous release cycle
 - Compiler optimization is not recommended based on experience
   from the previous release cycle 

Detailed results follow. 



====================== 
Static Analysis: 
====================== 

The Forcheck static analyzer warns of no additional coding errors in the
current version of pre-release-patches. The total number of errors and
warnings has remained the same since the last release. 

The total number of info messages has increased since the last release, 
emphasizing the need for more thorough static analysis testing by developers
prior to merging new code with the development_branch. Most of the info
messages refer to unreferenced variables and dummy arguments that usually
result from copying existing blocks of code which already contain such 
untidy code.

====================== 
Regression testing: 
====================== 

Changes since the last revision have introduced some numerical
differences into the results. The numerical differences were anticipated
by developers, and no unintended effects were observed during testing. 

One revision introduced expected numerical differences into the results: 

 r4682: This commit patched a bug in cetc/hvacsim.F that caused erroneous
        reporting of the part load ratio for idealized cooling systems.
        As a result, numerical differences were reported for the following
        test cases: 
        
        ashp_cool_auto_conv_summer (diff in .csv output only)
        ashp_cool_auto_conv_winter (diff in .csv output only)
        ashp_cool_cont_conv_summer (diff in .csv output only)
        ashp_cool_cont_conv_winter (diff in .csv output only)
        ccht_gcep_summer
        ccht_gcep_winter (diff in .csv output only)
        ccht_gshp.H4_summer
        ccht_gshp.H4_summer_MooreModel
        ccht_gshp.H4_winter (diff in .csv output only)
        ccht_gshp.HS_summer
        ccht_gshp.HS_winter (diff in .csv output only)
        ccht_gshp.SL_summer
        ccht_gshp.SL_winter (diff in .csv output only)
        ccht_gshp.V1_summer
        ccht_gshp.V1_winter (diff in .csv output only)
        
        
One revision modified an existing test_suite model:

 r4722: This commit updated the complex_fenestration model for testing
        a new shading control facility (see r4736 for details). 
        Numerical predictions with the release candidate could not be
        compared to the previous release as a result of the new
        functionality. 

        
One revision resulted in an increase in efficiency:

 r4841: This commit modified a loop indexing scheme to significantly reduce
        loop iterations and operations on zero value arrays which were never
        used anyway. Only models with a radiant floor plant component are
        affected by this change. 
        
        
Two new models were introduced to the test suite: 
 
 cellular_miso: This test model supports the implementation of a new global
        controller. This controller senses the state of control loops in 
        plant and flow domain and actuates the actuator of one control loop
        in the plant or flow domain. Hence emulating the action of a simple 
        MISO (multiple input, single output) controller. See r3607 for details.
 
 sloped_obstr: This test model supports the implementation of additional
        obstruction rotations and generalized six-sided obstructions as
        described in r4710. 


 
====================== 
Portability testing: 
====================== 

NRCan will no longer perform cross-platform and cross-compiler tests. Instead,
an archive of results built on Linux using the current reference compiler
is available in the /tester/testing_results folder. Users on other platforms and
compiler sets will assume responsibility for testing their platform against
this archive and examining any differences. 

This can be done using the tester.pl script located in /tester/scripts. For 
instance, to compare the results between a local bps binary and the reference
archive, issue the command:

$ ./tester.pl /path/to/test/bps --historical_archive 
       ref_platform_linux_gcc_4_1_2_results_archive_11_8.tar.gz
    
For a more detailed description of tester.pl usage, issue the command:
$ ./tester.pl --help
    
    
=========================== 
Problematic Test Cases* 
=========================== 
*(based on experience from last release cycle)

-  bat_life_no_control / h2-ctrl
   ------------------------------------------------- 
   This test case consistently produces large errors across all
   platforms except Linux GCC 4.1.2 and Cygwin GCC 4.1.2.
   
   
-  bld_hc_ISO15099 / HC
                   / HC_no-ISO
   ------------------------------------------------- 
   These test cases consistently produce large errors across all platforms.
   
-  cellular_offices / office_operations
   ------------------------------------------------- 
   This test case consistently produces significant (~5%) errors across all
   platforms. 
   
   
-  idealized_hvac / ashp_cool_auto_conv_winter
                  / ashp_cool_cont_conv_winter
   -------------------------------------------------   
   While they run correctly in every other compiler, these test cases appear
   to produce numerical exceptions in the SOLARIS / CC f90 platform, 
   reporting temperature values of -99.9(oC).
